Working with the File Channel data files

The File Channel stream data files are encoded in an internal binary format. For full-load operations, the File Channel binary files contain packed data records for each of the table records and an end-of-file (EOF) record. For change-processing operations, the file contains:

• A packed data record for each DDL and/or DML change.
• A begin-load-table record with the stream name that marks the beginning of table loading.
• A packed table-definition record with the table metadata. These records come before each DDL and begin-load-table record.

You do not need to work directly with the file-channel files, however if you find it necessary to work with them they are located in the File-Channel Directory Structure described below.

File-Channel directory structure

The file-channel directory contains the following files and folders:

• s_msgs: This folder contains messages sent from the source side to the replication server on the remote target side.

  Messages are removed from this folder at the source side when an acknowledgment message is received stating that the file was transferred successfully or possibly with a timeout.

  Messages are removed from this folder at the target side after they are read.

  This folder contains the following files:

  • s_msgs/xxxxxxxx.fcm: This file contains a JSON message from the source side to the target side.
  • yyyymmddhhMMsss.mtd: This file contains the captured tables list.
• s_status: This folder contains status updates from the source side to the target side. Status updates appear as a fixed name file that is periodically updated. This file lists the last processed target status file. It receives the t_status/cccccccc.fcs file. These files are deleted when the file-channel source endpoint finishes reading the file. You can configure the file-channel source to keep the files, if necessary. See Setting advanced connection properties for more information.

• t_status: This folder contains status updates from the target side to the source side. Status updates appear as an infinite set of data files that are created according to a specific schedule. These files are sent from the target by the source. The folder contains also a fixed name file that is updated with the last created status file name. It contains the following file:

  • t_status/cccccccc.fcs: This is a file channel status file (.fcs) where the file name is a hexadecimal counter of length 8. These files will be transferred in order with the lower numbers transferred first. If you need to view them, you should order them by timestamp because alphabetical ordering will not be consistent with the hexidecimal name.

    File channel status files are deleted by the source after being read and by the target when source status file indicates that this file was already processed.

    You can configure the maximum amount of time that the files are kept before a new file is created as well as the maximum file size for each file. The minimum file size is 50 MB.

    For more information, see Setting advanced connection properties.

• streams/<stream-name>: This folder contains stream sub-folder, one sub-folder per stream. A stream represents a finite or infinite set of data files being sent from the source to the target. The file channel allows creating and destroying named streams dynamically. For example, there can be a fixed-named stream cdc (streams/cdc) and there could be a dynamically created stream loadXXXXXXXX that can be removed at the source side when a status update from the target is received (for example, when processing completed) in the t_status folder.

  You can configure the maximum number of streams and the maximum disc space for each stream. For more information, see Change Processing.

  This folder contains the following file:

  • streams/<stream-name>/cccccccc.fcd: This is a file channel data file (.fcd) where the file name is a hexadecimal counter of length 8. These files are processed at the target in order or in parallel depending on the case. However, the files are transferred in order with the lower numbers transferred first.

    File channel data files are deleted by the source when transferred successfully and by the target when processed.

    You can configure the maximum amount of time that the files are kept before being creating a new file and the maximum file size for each file. The minimum file size is 10 MB and the minimum time that a file is kept is 5 seconds.